Apparatus and method for processing call services in mobile terminal

ABSTRACT

An apparatus and method for processing call services in a mobile terminal are provided. The method for processing call services in a mobile terminal includes entering into, when a call is generated, a call handling mode, recognizing and analyzing voice signals sent and received in the call to produce speech analysis information, detecting a state change of the mobile terminal using a sensing unit to produce user behavior information, and predicting, when a state change of the mobile terminal is detected, an application corresponding to the speech analysis information, and activating the application as a follow-up service.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 13/741,986, filed on Jan. 15, 2013, which has issued as U.S. Pat.No. 9,264,869 on Feb. 16, 2016 and claimed the benefit under 35 U.S.C.§119(a) of a Korean patent application filed on Jan. 18, 2012 in theKorean Intellectual Property Office and assigned Serial No.10-2012-0005755, the entire disclosure of which is hereby incorporatedby reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to call service processing in a mobileterminal. More particularly, the present invention relates to a callservice processing apparatus and method that can offer a follow-upaction in advance during an ongoing call by analyzing the phoneconversation in a mobile terminal.

2. Description of the Related Art

In addition to voice call handling, an advanced mobile terminal mayperform various functions including speech recognition and multimediaprocessing. The mobile terminal may employ various sensors to controldifferent functions. A mobile terminal may be controlled by verbalcommands input through speech recognition. To save battery power andprevent malfunction, a mobile terminal equipped with a touchscreen maybe controlled on the basis of sensing signals collected by varioussensors so as to disable display and touch input functions while themobile terminal is brought close to the ear during a call.

In some cases, during an ongoing call, the user of a mobile terminal mayhave to perform another action according to phone conversationinformation. For example, during an ongoing call, the user may have tocheck schedules and/or messages or enter keys to initiate a specificfunction. Having to perform a different action during a call may troublethe user and disrupt the phone conversation. However, existing mobileterminals provide only a limited function to handle this problem.

Hence, a need exists for a system and method that proposes a user actionin advance during an ongoing call on the basis of phone conversationinformation, counterpart information and/or the state of the mobileterminal.

The above information is presented as background information only toassist with an understanding of the present disclosure. No determinationhas been made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the present invention.

SUMMARY OF THE INVENTION

Aspects of the present invention are to address the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the present inventionprovides a call service processing apparatus and method that can offer afollow-up action in advance during an ongoing call in a mobile terminalby analysis of the phone conversation with the counterpart.

The present invention also provides a call service processing apparatusand method for a mobile terminal that propose a user action in advanceduring an ongoing call on the basis of phone conversation information,counterpart information and/or the state of the mobile terminal.

To achieve this, during an ongoing call in a mobile terminal, phoneconversation information is analyzed through speech recognition and afollow-up action for the user is proposed in advance according to theanalysis results. Here, the follow-up action may be proposed inconsideration of information on the counterpart of the call and/or thecurrent state of the mobile terminal in addition to speech recognitionresults.

In accordance with an exemplary embodiment of the present invention, amethod for processing call services in a mobile terminal is provided.The method includes recognizing and analyzing voice signals sent andreceived in a call, displaying, when the voice signals exhibit anAutomatic Route Selection (ARS) pattern, a keypad for enteringalphanumeric information, and providing an ARS call service usinginformation entered through the keypad.

In accordance with another exemplary embodiment of the presentinvention, an apparatus for processing call services in a mobileterminal is provided. The apparatus includes a communication unit forsending and receiving voice signals during a call, a sensing unit forsensing a state change of the mobile terminal, a control unit forrecognizing, during a call, voice signals, producing speech analysisinformation on the basis of speech recognition results, predicting, whena state change of the mobile terminal is detected by the sensing unit,an application corresponding to the speech analysis information, andactivating the application as a follow-up service, and a display unitfor outputting the follow-up service.

In accordance with another exemplary embodiment of the presentinvention, a method for processing call services in a mobile terminal isprovided. The method includes entering into, when a call is generated, acall handling mode, recognizing and analyzing voice signals sent andreceived in the call to produce speech analysis information, detecting astate change of the mobile terminal using a sensing unit to produce userbehavior information, and predicting, when a state change of the mobileterminal is detected, an application corresponding to the speechanalysis information, and activating the application as a follow-upservice.

Other aspects, advantages, and salient features of the invention willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses exemplary embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainexemplary embodiments of the present invention will be more apparentfrom the following description taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a block diagram of a mobile terminal according to an exemplaryembodiment of the present invention;

FIG. 2 is a flowchart of a method for suggesting follow-up actionsthrough recognition of an ongoing conversation in a phone call accordingto another exemplary embodiment of the present invention;

FIG. 3 is a flowchart of a procedure for suggesting a follow-up actionon the basis of speech recognition results (or a voice command of theuser) in a phone conversation according to an exemplary embodiment ofthe present invention;

FIG. 4 illustrates a configuration of a control unit of the mobileterminal, which is designed to propose a follow-up action in a call onthe basis of phone conversation information, counterpart information oruser behavior according to an exemplary embodiment of the presentinvention;

FIG. 5 is a flowchart of a procedure for suggesting a follow-up actionin a call performed by a mobile terminal having a control unit with aconfiguration of FIG. 4 according to an exemplary embodiment of thepresent invention;

FIG. 6 is a flowchart of a procedure for determining a follow-up actionin a call performed by a follow-up activity handler of the control unitusing received analysis results according to an exemplary embodiment ofthe present invention;

FIGS. 7A and 7B illustrate activation of a scheduling function during acall according to an exemplary embodiment of the present invention;

FIGS. 8A and 8B illustrate activation of a phishing alert functionduring a call according to an exemplary embodiment of the presentinvention; and

FIGS. 9A and 9B illustrate activation of a follow-up action in anAutomatic Route Selection (ARS) call according to an exemplaryembodiment of the present invention.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of exemplaryembodiments of the invention as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the embodiments described hereincan be made without departing from the scope and spirit of theinvention. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of theinvention. Accordingly, it should be apparent to those skilled in theart that the following description of exemplary embodiments of thepresent invention is provided for illustration purpose only and not forthe purpose of limiting the invention as defined by the appended claimsand their equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

The present invention relates to an apparatus and method that canpropose a follow-up action in advance during an ongoing call in a mobileterminal by identifying user needs. To achieve this, conversationinformation is collected through speech recognition during a call.Information on state or position changes of the mobile terminal iscollected. The situation and the intention of the user are identified onthe basis of the collected information. A follow-up action or activityis determined according to the identified situation and intention, andis presented to the user.

In an exemplary embodiment of the present invention, during a call, thephone conversation between the user and counterpart user is analyzedthrough speech and voice recognition, and a follow-up action or activityfor the user is predicted on the basis of the analysis results andpresented to the user. When the user changes the position of the mobileterminal during the call, this may indicate the intention of the user toperform an action other than conversation. Hence, one or more useractions may be effectively proposed in advance during an ongoing call onthe basis of any of conversation analysis results, position changes ofthe mobile terminal, counterpart user information, and the like.

FIG. 1 is a block diagram of a mobile terminal according to an exemplaryembodiment of the present invention.

Referring to FIG. 1, a communication unit 120 of the mobile terminalperforms wireless communication with a base station or another mobileterminal. The communication unit 120 may include a transmitter forup-converting the frequency of a signal to be transmitted and amplifyingthe signal, and a receiver for low-noise amplifying a received signaland down-converting the frequency of the received signal. Thecommunication unit 120 may further include a modulator for modulating asignal to be transmitted and forwarding the modulated signal to thetransmitter, and a demodulator for demodulating a signal receivedthrough the receiver. Modulation and demodulation may be conductedaccording to Long Term Evolution (LTE), Wideband Code-Division MultipleAccess (WCDMA), Global System for Mobile Communications (GSM), Wi-Fi,Wireless Broadband (WiBro), Near Field Communication (NFC) or Bluetooth.

A control unit 100 controls the overall operation of the mobileterminal. More particularly, the control unit 100 may include a voiceanalyzer to analyze phone conversation information during a call, andmay predict a follow-up action of the user and present the predictedfollow-up action to the user during the call.

A memory unit 110 may include a program area to store programs foroperating the mobile terminal and for realizing functions of the presentinvention, and a data area to store tables used for terminal operationand data generated during program execution. More particularly, thememory unit 110 may store a speech recognition table for callconversation analysis and recognition.

A voice processing unit 130 processes voice signals sent and receivedthrough the communication unit 120. A sensing unit 140 detects statechanges of the mobile terminal caused by the behavior and movement ofthe user. The sensing unit 140 may include an ambient light sensor tosense illumination intensity and a camera module to capture images ofexternal objects. The sensing unit 140 may further include anacceleration sensor, geomagnetic sensor and gyro sensor to sense themovement and posture of the mobile terminal.

A display unit 150 may include a Liquid Crystal Display (LCD) or OrganicLight-Emitting Diode (OLED) panel to display data and images generatedby programs under control of the control unit 100. An input unit 160 mayinclude a touch panel that detects a touch gesture of the user and sendsa corresponding touch signal to the control unit 100. The input unit 160and the display unit 150 may be combined as a touchscreen.

In the mobile terminal having the above configuration, during a call,the control unit 100 applies speech recognition to voice signalsprocessed by the voice processing unit 130 and analyzes the callconversation on the basis of the speech recognition results. The controlunit 100 analyzes the call conversation to present a follow-up action oractivity using a speech recognition table stored in the memory unit 110.More particularly, the call conversation is analyzed to extractinformation related to a schedule or time, phone number, location,person and the like. The extracted information is used to predict afollow-up activity. Here, a follow-up action or activity refers to auser function of the mobile terminal that may be invoked by the user inconversation with a counterpart user during a call. Examples of afollow-up action during a call may include schedule checking andplanning, message browsing and viewing, recording, spam registration,and activation of the input unit.

Sometimes, during an ongoing call, the user of a mobile terminal mayhave to perform another action (e.g., schedule checking and planning,memo writing, phonebook searching, and the like) according to phoneconversation information. In this case, the user is inconvenienced byhaving to suspend the conversation, perform a needed action or function,and then resume the conversation. Hence, it is preferable for the mobileterminal to analyze the phone conversation and to propose a neededaction or function in advance.

In an exemplary embodiment, the control unit 100 includes a speechanalyzer to recognize the conversation during a call. When presetinformation is detected in the conversation, the control unit 100automatically activates a corresponding function during the call.

In recent years, speech recognition technology has been rapidlyadvanced. Beyond speech-to-text conversion or keyword recognition,speech recognition software may understand the context of a conversationmuch like a human being may understand. Exemplary embodiments of thepresent invention provide a necessary service to the user during a callthrough the use of speech recognition. Speech recognition may beperformed at the mobile terminal using a proprietary solution or a thirdparty solution, or performed through interworking between the mobileterminal and a server. In the following description, the control unit100 is assumed to include a speech analyzer. The speech analyzer maypresent a follow-up action to be performed by the mobile terminalthrough speech recognition of the call conversation. The speech analyzermay use keyword recognition, context understanding and/or conversationpattern recognition.

The speech analyzer of the control unit 100 analyzes a phoneconversation to identify the activity type and other key informationrelated to the activity type. Here, examples of key information mayinclude time information, number information, location information,personal information, and the like.

A speech recognition table summarizing speech recognition results isillustrated in Table 1. The speech recognition table illustrates variousparameters used to derive a follow-up action in a phone conversation.

TABLE 1 Key Example phrases Activity type information Activity Do youhave time for Schedule Time, day Check schedule at lunch time lunchtomorrow? tomorrow. When no appointment is present at that time,schedule an appointment with the counterpart. Hello, this is AAA KeypadOutput a keypad. To maintain Telecom. Please enter a call connection inuse of a desired service number. keypad, increase volume of call If youwant to discuss or speakerphone. call quality . . . Let me know thephone Phonebook, text Personal Find phone number of Kim BB number of KimBB? message information from phonebook, and help the user to send a textmessage. Have you checked an Email Personal Check email, find an emailsent email from BBB information to or received from the corporation?counterpart, or find an email Please check emails! corresponding topersonal information. Hello? Who is this Phonebook, Find similar numberin speaking? search call phonebook, check call list for history phonenumber Hello. This is YYY of Spam Phone Register caller number as spamXX credit card. Please registration number allow me to introduce newservices to you. Hello. You won a prize Phishing alert, Check securityfor the caller in an event. Please enter search the Web number, warnpossibility of registration number for for the caller phishing identityconfirmation? number Darling, do you Schedule, check Day or date Searchaddress book for remember what day anniversary anniversary date of thetoday is? dates counterpart, check schedules Hey, when is the Schedule,check Personal Provide information on persons anniversary of the deathaddress book. information, of ABC's father? event information How areyou? This is Check address Personal Provide information on persons CCS.Long time no talk. book, check information so as to help management ofHow have you been? Simple personal connections Notification Service(SNS) Do you have a pen and Memo, Conversation Switch to memo mode,increase some paper? Can you speakerphone, volume of call orspeakerphone write a memo? recording

In an exemplary embodiment, the user may issue a voice command to themobile terminal during a call. In this case, the speech analyzer of thecontrol unit 100 may fail to predict a follow-up action owing tomultiple sources of sounds, e.g., a phone conversation and voicecommands. Processing of a voice command may be separated from analysisand recognition of the phone conversation. A preset key may be used tonotify a voice command during a call, and it is preferable not totransmit a voice command to the counterpart user. For example, during acall, when the user enters a preset key and pronounces “Galaxy, checkschedules” as a voice command, the control unit 100 may block forwardingof the voice command to the communication unit 120 and perform aschedule checking function. When voice commands are used as above, astraightforward speech recognition table may be formed in the memoryunit 110. Hence, when the user issues a voice command during a call, thecontrol unit 100 may recognize the voice command and perform acorresponding function.

FIG. 2 is a flowchart of a method for suggesting follow-up actionsthrough recognition of an ongoing conversation in a phone call accordingto another exemplary embodiment of the present invention.

Referring to FIG. 2, when a call is placed by the user or a call isreceived from a counterpart user, the control unit 100 of the mobileterminal enters into a call handling mode at step 211. Here, a callconnection is established through the communication unit 120 between themobile terminal and the counterpart mobile terminal. The voiceprocessing unit 130 processes voice signals sent and received throughthe communication unit 120. In the call handling mode, the control unit100 performs speech recognition to analyze the phone conversation atstep 213, determines a follow-up action on the basis of speechrecognition results, and performs a corresponding function at step 215.At step 213, the control unit 100 may recognize the phone conversationbetween the user and counterpart user or a voice command from the user.At step 217, the control unit may determine whether the call hasterminated. If the call has not terminated, the control unit may returnto call handling mode and perform speech recognition.

Step 215 for presenting a follow-up action is described in FIG. 3.

FIG. 3 is a flowchart of a procedure for suggesting a follow-up actionon the basis of speech recognition results (or a voice command of theuser) in a phone conversation according to an exemplary embodiment ofthe present invention.

Referring to FIG. 3, the control unit 100 analyzes speech recognitionresults at step 311. Table 1 illustrates speech recognition resultsobtained by recognizing what the counterpart user says. Here, speechrecognition results may include date information, message information,counterpart information (e.g., personal information, number information,and the like) location information, and the like. One or more pieces ofsuch information may be recognized.

Here, date information may be given in an explicit form (numerical date)and/or in an implicit form like “today”, “tomorrow”, “yesterday”, “lunchtime”, “evening”, “AM”, “PM” and the like. Personal information refersto information related to persons enrolled in a phonebook, and mayinclude a user name, phone number, an email address, SNS information andthe like. Location information may include a city name, region name,address, building name, road name and the like. Message information mayinclude information related to a Short Message Service (SMS) message,Enhanced Message Service (EMS) message, Multimedia Messaging Service(MMS) message, SNS message, instant message and the like.

In the case where date information is present in the speech recognitionresults at step 313, the control unit 100 retrieves a list of scheduleitems stored in the memory unit 110 at step 315, and displays the listof schedule items at step 317. When an enrollment request is issued atstep 319, the control unit 100 adds a schedule item containing the nameof the counterpart user, and indicated time and location to the list atstep 321. When a search request is issued at step 323, the control unit100 searches the list for a schedule item associated with the dateinformation and displays the search result at step 325. Here, the usermay make a specific request through a touch gesture or through a voicecommand.

In the case where message information is present in the speechrecognition results at step 331, the control unit 100 searches formessages at step 333. Here, as described before, messages may include anemail message, an SMS message, an MIMS message, an SNS message, aninstant message and the like. Messages may be searched using personalinformation or using both date information and personal information. Thecontrol unit 100 displays a list of searched messages on the displayunit 150 at step 335. When one of the displayed messages is selected atstep 337, the control unit 100 displays the selected message at step339. Here, message selection may be achieved through a touch gesture orthrough a voice command.

In the case where key information requiring storage or recording ispresent in the speech recognition results at step 341, the control unit100 may store the key information in the memory unit 110 or record thephone conversation and store the recording at step 343. This step may beinvoked when certain words, such as “memo” or “record,” are recognizedin the conversation, when the counterpart user is associated with aprevious call requiring storage or recording, when the conversation isrelated with a legal matter requiring recording and the like.

In the case where a spam pattern is present in the speech recognitionresults and the phone number of the counterpart user is not enrolled atstep 351, the control unit 100 may regard the call as spam and mayregister the phone number and personal information of the counterpartuser as spam at step 353. In the case where entry of a code number isrequested and the phone number of the counterpart is in an AutomaticRoute Selection (ARS) number format at step 361, the control unit 100regards the call as an ARS call and displays a keypad for entering acode number at step 363.

As described above, an exemplary mobile terminal of the presentinvention performs speech recognition of the conversation during a call,predicts a user action (follow-up activity) on the basis of speechrecognition results, and executes a function supporting the predicteduser action in advance. Hence, the user of the mobile terminal does nothave to directly invoke a different function during a call and may paymore attention to the ongoing conversation.

To present more appropriate and versatile follow-up actions during acall, other types of information such as personal information related tothe counterpart user and behavioral changes of the user in the phoneconversation may be utilized in addition to speech recognition results.

FIG. 4 illustrates a configuration of the control unit of the mobileterminal, which is designed to propose a follow-up action in a call onthe basis of phone conversation information, counterpart information anduser behavior according to an exemplary embodiment of the presentinvention.

Referring to FIG. 4, a speech analyzer 410 identifies an activity thatmay be performed during a call on the basis of speech recognitionresults, and sends key information corresponding to the identifiedactivity to a follow-up activity handler 440. When the user issues avoice command for a follow-up action during the call, the speechanalyzer 410 may send key information corresponding to the voice commandto the follow-up activity handler 440.

A counterpart information analyzer 420 collects information on thecounterpart user of a call. For a counterpart user, information on thefrequency of calls and activity type involved in the phone conversationmay be collected. Collected counterpart information is sent to thefollow-up activity handler 440.

A state change analyzer 430 collects information on posture and thestate of the mobile terminal during a call. During a call, the mobileterminal may be placed close to the ear, a speakerphone may be used, orthe mobile terminal may be moved because of user action. Such movementand state changes may be detected through sensors. Collected statechange information is sent to the follow-up activity handler 440.

The follow-up activity handler 440 determines a follow-up action on thebasis of information received from the speech analyzer 410, counterpartinformation analyzer 420 and state change analyzer 430, and presents thefollow-up action in advance during the call.

In addition to the speech recognition table, the memory unit 110 mayfurther store an action-state table for analyzing state changes of themobile terminal due to user action, and a counterpart information tablefor analyzing information on a counterpart user of a call.

As described above, the speech analyzer 410 analyzes the conversationbetween the user and counterpart user during a call through speechrecognition. The speech analyzer 410 may produce analysis results asillustrated in Table 1. The counterpart information analyzer 420collects information on the counterpart user. Counterpart informationmay be obtained from signaling data from the base station, a phonenumber, and information attached to an email, an SNS, an instantmessage, and the like. Counterpart information may be obtained from theInternet through the communication unit 120. The state change analyzer430 may collect information on state changes of the mobile terminalduring a call through the sensing unit 140.

The follow-up activity handler 440 determines a follow-up action bycombining analysis results from the speech analyzer 410, counterpartinformation analyzer 420 and state change analyzer 430, and presents thefollow-up action in advance during the call. Here, each of the speechanalyzer 410, counterpart information analyzer 420 and state changeanalyzer 430 may score the analysis results, and the follow-up activityhandler 440 may determine a follow-up action using numerical scores.

The speech analyzer 410 may analyze the conversation, collect variousinformation such as date information, personal information, locationinformation, number information and event information, score thecollected information items, and output the scored results to thefollow-up activity handler 440. The speech analyzer 410 may utilize aspeech recognition table illustrated in Table 1.

The counterpart information analyzer 420 may use the phone number of acounterpart user in a call to identify a probable action. For example,in an ARS call, the counterpart information analyzer 420 may identifythe use of a keypad as a probable action. When a call with an unknownphone number is received, the counterpart information analyzer 420 maydetermine to search the call history for the phone number, search thephonebook for a similar phone number, or to search a black list for thephone number. When a received call is related to business (e.g., a callfrom a business partner), the counterpart information analyzer 420 maydetermine to find transaction documents, memos, email messages relatedto the corresponding company, and the like.

Table 2 illustrates a counterpart information table.

TABLE 2 Counterpart Probable action ARS Use of keypad, use ofspeakerphone, call recording Enrolled user-business Search emails,schedules, recordings, partner memos, or contact information(department, rank or the like) Enrolled user-relative, Search ortransmit schedules, memos, friend contact information (birthday,anniversary or the like) Unenrolled phone number Find similar phonenumber, make memo, record or schedule, spam registration, heck callhistory, warn phishing Enrolled user-legal issue Recording, memowriting, (contract, lawsuit) schedule making Enrolled user-userselection Check user settings

The state change analyzer 430 uses sensing signals from the sensing unit140 to monitor the state of the mobile terminal and to identify ameaningful change in a user's behavior, which may indicate a probablefollow-up action. In addition to the sensing unit 140, the state changeanalyzer 430 may identify a change in a user's behavior through externalbuttons and through the input unit 160. The state change analyzer 430may identify a change in a user's behavior with reference to presetactions or movements.

Here, the actions may be preset gestures or natural actions of the userduring a call. For example, a gesture of laterally shaking the mobileterminal two times may be preset as a request for a specific function;and the user may laterally shake the mobile terminal two times to invokethe function. In some cases, natural actions of the user during a phoneconversation may be interpreted as a request for a specific function.For example, the user in phone conversation may move the mobile terminalfrom a position close to the ear to another position in front of theeyes in order to view some information item. The sensing unit 140 maydetect a large change in illumination intensity (ambient light sensor),detect rotational movement and stoppage of the mobile terminal(acceleration sensor), detect a change in the position of the mobileterminal, and detect the face or eye of the user (camera module). Suchchanges in the position and state of the mobile terminal may indicate anintention of the user wishing to find some information during a call. Asanother example, when the user uses an earphone during phoneconversation, the user may carry the mobile terminal in various ways.However, a user seldom holds a mobile terminal in front of their eyesduring phone conversation. When the user places the mobile terminal infront of their eyes, rotational movement and stoppage of the mobileterminal may be detected and the state of the user may be identifiedthrough the camera module.

The state change analyzer 430 may identify state changes of the mobileterminal during a call using an action-state table illustrated in Table3.

TABLE 3 Determination Scenario for Scenario during call conditionsservice request Determination conditions (change) Talking with lightsensor: low Placing mobile light sensor: high light intensity terminalclose to the light intensity terminal before movement sensor: detectpreset ear movement sensor: the eye for visual action (rotation withlarge motion) little movement information camera: detect face (eye)camera: Off Talking with light sensor: low light intensity terminalclose to movement sensor: detect preset the ear, checking action (e.g.shaking) information with button input: enabled (e.g. power audiobutton) Talking with light sensor: high To check movement sensor: detectpreset terminal detached light intensity information, move action(stoppage or little motion, from the ear- movement sensor: terminalbefore rotation with large motion, various postures little movement theeye so that stoppage) are possible, but camera: On terminal is parallelcamera: detect face (eye) terminal is seldom touch input: No with theface for touch gesture sensor: detect preset in front of the face visualinformation action (stoppage or little motion, rotation, stoppage atgiven angle) camera: detect face (eye) touch input: enabled buttoninput: enabled (e.g.: power button, home button) Talking with anearphone: To check movement sensor: detect preset earphone- connectedinformation, move action (stoppage or little motion, various postureslight sensor: terminal before large motion, stoppage) are possible, butOn/Off the eye so that camera: detect face (eye) terminal is seldommovement sensor: terminal is parallel touch input: enabled in front ofthe face little movement with the face for button input: enabled (e.g.:power camera: On visual information button, home button) Talking throughBluetooth: To check movement sensor: detect preset Bluetooth- connectedinformation, action (stoppage or little motion, in most cases, lightsensor: manipulate large motion, stoppage) terminal is detached On/Offdetached terminal camera: detect face (eye) from the user movementsensor: button input: enabled (e.g.: power little movement button, homebutton) camera: On touch input: enabled Talking while light sensor: highMove terminal movement sensor: detect preset manipulating the lightintensity before the eye for action (e.g. shaking) screen touch input:Yes visual information button input: enabled Using speakerphone movementsensor: To check movement sensor: detect preset with terminal placedstoppage information, action (e.g. jogging, shaking) at a fixed locationtouch input: No manipulate camera: detect face (eye) button input: Noterminal touch input: enabled An incoming call Come to answer touchinput: enabled arrives the call button input: enabled movement sensor:detect preset action camera: detect face (eye)

FIG. 5 is a flowchart of a procedure for suggesting a follow-up actionin a call performed by the mobile terminal having a control unit withthe configuration of FIG. 4 according to an exemplary embodiment of thepresent invention.

Referring to FIG. 5, the control unit 100 enters into a call handlingmode at step 511. In the call handling mode, the control unit 100analyzes information on the counterpart user of the call at step 513.Here, the control unit 100 may check whether the phone number of thecall is enrolled in the mobile terminal. When the phone number isenrolled, the control unit 100 may check whether the counterpart useris, e.g., a relative, a friend, or a business partner. When the phonenumber is not enrolled, the control unit 100 may collect information onthe phone number through an external network, like the Internet, or maycheck whether the phone number is an ARS number. After analyzing thecounterpart user, the control unit 100 performs speech recognition toanalyze the phone conversation and to extract information for probablefollow-up activities at step 515. Here, the speech analyzer 410 collectsinformation for probable follow-up activities during the call. Thecontrol unit 100 monitors the state of the mobile terminal through thesensing unit 140 to identify a user behavior for which the control unitneeds further information during the conversation at step 517.

The counterpart information, speech analysis information and userbehavior information collected through steps 513 to 517 may beincomplete in themselves. This incompleteness may be alleviated bycombining these pieces of information together. To achieve this,information items collected by each of the speech analyzer 410,counterpart information analyzer 420 and state change analyzer 430 maybe scored according to probability. That is, a high score may beassigned to a highly probable item. Each of the analyzers 410, 420 and430 may assign different scores to multiple information items. Thecounterpart information, speech analysis information and user behaviorinformation are sent together with assigned scores to the follow-upactivity handler 440. The control unit 100 determines a follow-up actionon the basis of the counterpart information, speech analysis informationand user behavior information, and presents the follow-up action duringthe call at step 519. A follow-up action is determined using items ofthe analysis results with scores higher than a threshold.

FIG. 6 is a flowchart of a procedure for determining a follow-up actionin a call performed by the follow-up activity handler of the controlunit using received analysis results according to an exemplaryembodiment of the present invention.

Referring to FIG. 6, the control unit 100 obtains counterpartinformation, speech analysis information and user behavior informationthrough the speech analyzer 410, counterpart information analyzer 420and state change analyzer 430 at step 611. Here, items of the analysisresults are scored by the corresponding analyzers 410 to 430. Thecontrol unit 100 extracts a list of probable activities on the basis ofthe obtained information, and sums up the scores assigned to informationitems associated with the probable activities at step 613. The controlunit 100 checks whether the total score for one probable activity isgreater than a preset threshold at step 615. When the total score forone probable activity is greater than the preset threshold, the controlunit 100 determines the probable activity to be a follow-up action(service or function) and outputs the follow-up action to the displayunit 130 during the call at step 617.

FIGS. 7A and 7B illustrate activation of a scheduling function during acall according to an exemplary embodiment of the present invention.

Referring to FIG. 7A, when an incoming call is received at step 711, thecontrol unit 100 notifies call reception. When the call is answered bythe user at step 713, the control unit 100 relays a phone conversationbetween the user and the counterpart user at step 715. Upon callreception, the counterpart information analyzer 420 may check whetherthe phone number of the call is enrolled in the mobile terminal. Whenthe phone number is enrolled, the counterpart information analyzer 420examines the frequency of calls or communications with the counterpartuser and assigns a corresponding score to the counterpart information.During the call, the speech analyzer 410 performs speech recognition toanalyze the phone conversation between the user and counterpart user.When the counterpart user talks about a schedule event with a given dateat step 717, the speech analyzer 410 generates corresponding dateinformation. In addition, the user may move the mobile terminal from theear to the eye and may manipulate the mobile terminal to check plannedschedules at steps 719 and 721. Here, the state change analyzer 430generates user behavior information on the basis of an output of thesensing unit 140. A user behavior may be detected through sensors of thesensing unit 140, such as an ambient light sensor, and movement sensor(e.g., an acceleration sensor, a geomagnetic sensor, a gyro sensor, andthe like).

After the generation of counterpart information (i.e., regarding anenrolled counterpart user), speech analysis information (e.g., includingdate information) and user behavior information, the follow-up activityhandler 440 determines a follow-up action by combining these pieces ofinformation and outputting the follow-up action to the display unit 150at step 723. For example, when the date information is for tomorrow'slunch appointment, the control unit 100 may search schedule items storedin the memory unit 110 for noon tomorrow and display the search resulton the display unit 150 as shown in FIG. 7B. That is, when a scheduleevent with a desired date and hour is detected in the phoneconversation, the control unit 100 may check existing planned scheduleitems with the date and hour, and display the checked result as afollow-up action for the user.

In FIG. 7A, the counterpart user has proposed tomorrow's lunchappointment during the phone conversation, thereby generating dateinformation. Then, the control unit 100 predicts a follow-up action ofschedule checking on the basis of speech recognition results, andinvokes a scheduling program for the user immediately after the usermoves the mobile terminal to another position. More specifically, thecontrol unit 100 analyzes the conversation through speech recognition.When the counterpart user proposes an appointment with a specific date,the control unit 100 predicts a follow-up action of schedule checking.At the same time, the control unit 100 determines that the phone numberof the call is enrolled in the mobile terminal (i.e., an enrolledcounterpart user) and examines the frequency of calls, appointments orcommunications with the counterpart user. When the user moves the mobileterminal to another position to check existing schedules (e.g., themobile terminal is moved from a position close to the ear to a positionin front of the eyes), the control unit 100 detects movement of themobile terminal (i.e., user behavior) through the sensing unit 140. Thecontrol unit 100 may interpret this user behavior as a request forscheduling checking, and may invoke a scheduling program as shown inFIG. 7B, thereby shortening schedule search time. Hence, the controlunit 100 may present a follow-up action in advance during a call.

FIGS. 8A and 8B illustrate activation of a phishing alert functionduring a call according to an exemplary embodiment of the presentinvention.

Referring to FIG. 8A, when a call with an unknown phone number isreceived at step 811, the control unit 100 displays a call receptionindication. Upon call reception at step 813, the counterpart informationanalyzer 420 may determine that the phone number is an unenrollednumber, search information on the phone number on the Internet throughthe communication unit 120, and display the search result at step 815.The counterpart information analyzer 420 may examine the call orcommunication history to check whether the phone number is registered asspam. When the call is answered by the user, the control unit 100 relaysa phone conversation between the user and the counterpart user at step817. During the call, the speech analyzer 410 performs speechrecognition to analyze the phone conversation between the user andcounterpart user. When the counterpart user speaks phishing-patternwords at step 819, the speech analyzer 410 generates a phishing riskindication as speech analysis information. In addition, the user maymove the mobile terminal from a position close to the ear to a positionin front of the eyes in order to check a phishing attempt, and the statechange analyzer 430 may generate corresponding user behavior informationon the basis of an output of the sensing unit 140.

After generating counterpart information (e.g., related to an unenrolledcounterpart user), speech analysis information (e.g., a phishing riskindication) and user behavior information, the follow-up activityhandler 440 determines a follow-up action by combining these pieces ofinformation and outputting the follow-up action to the display unit 150at step 821. When speech analysis information indicates a phishing risk,the control unit 100 may output a phishing alert message to the displayunit 150 as indicated by reference numeral 850 of FIG. 8B. That is, whenthe counterpart user speaks phishing-pattern words, the control unit 100may output a phishing alert message in advance as a follow-up action.

When the phone number of the call is not enrolled, the control unit 100may search a stored phonebook for a similar phone number. When the phoneconversation is unrelated to a phishing pattern, the control unit 100may output similar phone numbers and associated call records as afollow-up service, as indicated by reference numeral 860 of FIG. 8B. Forexample, a corporation may use many similar phone numbers, and the samecorporate member may place a call with similar but different phonenumbers. To handle this case, when the phone number of a call is similarto an enrolled phone number, the control unit 100 may output similarphone numbers as a follow-up action for call reception, as indicated byreference numeral 860 of FIG. 8B.

FIGS. 9A and 9B illustrate activation of a follow-up action in an ARScall according to an exemplary embodiment of the present invention.

Referring to FIG. 9A, when a call is placed at step 911, the controlunit 100 notifies of the call generation. When the user responds to thecall, the control unit 100 provides a call service at step 913. Uponcall generation, the counterpart information analyzer 420 may checkwhether the call is an ARS call. As ARS phone numbers have a differentformat, the counterpart information analyzer 420 may distinguish an ARSphone number from a regular phone number. During the call, the controlunit 100 performs speech recognition to analyze whether the phoneconversation has an ARS pattern at step 915. The follow-up activityhandler 440 may then identify the call as an ARS call, and may present afollow-up action for the ARS call on the basis of analysis informationfrom the speech analyzer 410 and from the counterpart informationanalyzer 420. In the case of an ARS call, the control unit 100 mayoutput a keypad for entering alphanumeric information as a follow-upaction in response to a request of the counselor as shown in FIG. 9B.That is, the control unit 100 may output a keypad for enteringalphanumeric information requested by the counselor as a follow-upaction for ARS call connection. Thereafter, the control unit 100 mayenter requested information through the keypad and may continue to relayconversation in the ARS call.

As described above, in the case of an ARS call, an ARS phone number hasa format different from that of a regular phone number, and has a uniqueconversation pattern (e.g., “Hello. This is XX customer service center.Please enter a desired service number.”). Here, the counterpartinformation analyzer 420 sends an ARS call indication as counterpartinformation to the follow-up activity handler 440. During the call (theuser does not talk while the counterpart user speaks a request forentering a desired service number), the speech analyzer 410 sends an ARSpattern indication as speech analysis information to the follow-upactivity handler 440. When the user moves the mobile terminal from aposition close to the ear to a position before the eye to enter adesired number, the state change analyzer 430 sends corresponding userbehavior information to the follow-up activity handler 440. Then, thefollow-up activity handler 440 identifies a user's need for entering adesired number and outputs a keypad to the display unit 150 as afollow-up action as shown in FIG. 9B on the basis of ARS call indicationand movement of the mobile terminal before the eye sensed by the sensingunit 140.

In another exemplary embodiment, when, e.g., the user receives legaladvice, the mobile terminal may automatically record the conversation asa follow-up action or service. More specifically, when the phone numberof the counterpart user has a recording indication or is discovered asbeing a phone number of a legal service provider through Internetbrowsing, the counterpart information analyzer 420 sends a recordingindication as counterpart information to the follow-up activity handler440. When the conversation includes legal words, the counterpartinformation analyzer 420 sends a legal issue indication as speechanalysis information to the follow-up activity handler 440. Thereafter,the follow-up activity handler 440 may determine that the conversationis related to a legal matter, and may be important for the user, andthus may record the conversation as a follow-up action. When the call isended, the follow-up activity handler 440 may notify the user of therecording and request the user to confirm whether to store the recordedresults.

In exemplary embodiments of the present invention, a mobile terminal canpredict a user action and can execute an application supporting thepredicted user action in advance, during an ongoing call, on the basisof the phone conversation, counterpart information and/or the state ofthe mobile terminal. Hence, the user of the mobile terminal does nothave to directly invoke a different function during a call, and may paymore attention to the ongoing conversation. Consequently, the presentinvention can improve user experience by suggesting an action desired bythe user in advance.

While the invention has been shown and described with reference tocertain exemplary embodiments thereof, it will be understood by thoseskilled in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the presentinvention as defined in the appended claims and their equivalents.

What is claimed is:
 1. An electronic device comprising: a display; acommunication module; and a control module operatively coupled to thedisplay and the communication module, the control module configured to:control the communication module to establish a call connection inresponse to a call request; and control the display to display, if acounterpart associated with the call connection corresponds to aspecified counterpart, a keypad in relation with the call connection. 2.The device of claim 1, wherein the control module is further configuredto control the display to display the keypad if a movement of theelectronic device is detected.
 3. The device of claim 1, wherein thespecified counterpart is an automatic response service (ARS) subscriber.4. The device of claim 2, wherein the movement of the electronic deviceis caused by a preset gesture or a natural action of a user.
 5. A mobilecommunication device comprising: a display; a communication module; anda control module operatively coupled to the display and thecommunication module, the control module configured to: control thecommunication module to establish a call connection in response to acall request; and control the display to display, if the call connectioncorresponds to an automatic response service (ARS) call, a keypad inrelation with the call connection.
 6. The device of claim 5, wherein thecontrol module is further configured to control the display to displaythe keypad if a movement of the mobile communication device is detected.7. The device of claim 6, wherein the movement of the mobilecommunication device is caused by a preset gesture or a natural actionof a user.
 8. An apparatus comprising: a display; a communicationmodule; and a control module operatively coupled to the display and thecommunication module, the control module configured to: control thecommunication module to establish a call connection in response to acall request; and control the display to display, if a phone numberassociated with the call connection corresponds to a specified phonenumber, a keypad in relation with the call connection.
 9. The device ofclaim 8, wherein the control module is further configured to control thedisplay to display the keypad if a movement of the apparatus isdetected.
 10. The device of claim 9, wherein the specified phone numberis an automatic response service (ARS) number.
 11. A method comprising:establishing a call connection at an electronic device in response to acall request; and displaying, if a counterpart associated with the callconnection corresponds to a specified counterpart, a keypad in relationwith the call connection.
 12. The method of claim 11, furthercomprising: displaying the keypad if a movement of the electronic deviceis detected.
 13. The method of claim 11, wherein the specifiedcounterpart is an automatic response service (ARS) subscriber.
 14. Themethod of claim 12, wherein the movement of the electronic device iscaused by a preset gesture or a natural action of a user.
 15. A methodcomprising: establishing a call connection at a mobile communicationdevice in response to a call request; and displaying, if the callconnection corresponds to an automatic response service (ARS) call, akeypad in relation with the call connection.
 16. The method of claim 15,further comprising: displaying the keypad if a movement of the mobilecommunication device is detected.
 17. The method of claim 16, whereinthe movement of the mobile communication device is caused by a presetgesture or a natural action of a user.
 18. A method comprising:establishing a call connection at an apparatus in response to a callrequest; and displaying, if a phone number associated with the callconnection corresponds to a specified phone number, a keypad in relationwith the call connection.
 19. The method of claim 18, furthercomprising: displaying the keypad if a movement of the apparatus isdetected.
 20. The method of claim 19, wherein the specified phone numberis an automatic response service (ARS) number.